Sensing, interpreting, and anticipating human social behaviour in the real world

Low-level nonverbal social signals like glances, utterances, facial expressions and body language are central to human communicative situations and have been shown to be connected to important high-level constructs, such as emotions, turn-taking, rapport, or leadership. A prerequisite for the creation of social machines that are able to support humans in e.g. education, psychotherapy, or human resources is the ability to automatically sense, interpret, and anticipate human nonverbal behaviour. While promising results have been shown in controlled settings, automatically analysing unconstrained situations, e.g. in daily-life settings, remains challenging. Furthermore, anticipation of nonverbal behaviour in social situations is still largely unexplored. The goal of this thesis is to move closer to the vision of social machines in the real world. It makes fundamental contributions along the three dimensions of sensing, interpreting and anticipating nonverbal behaviour in social interactions. First, robust recognition of low-level nonverbal behaviour lays the groundwork for all further analysis steps. Advancing human visual behaviour sensing is especially relevant as the current state of the art is still not satisfactory in many daily-life situations. While many social interactions take place in groups, current methods for unsupervised eye contact detection can only handle dyadic interactions. We propose a novel unsupervised method for multi-person eye contact detection by exploiting the connection between gaze and speaking turns. Furthermore, we make use of mobile device engagement to address the problem of calibration drift that occurs in daily-life usage of mobile eye trackers. Second, we improve the interpretation of social signals in terms of higher level social behaviours. In particular, we propose the first dataset and method for emotion recognition from bodily expressions of freely moving, unaugmented dyads. Furthermore, we are the first to study low rapport detection in group interactions, as well as investigating a cross-dataset evaluation setting for the emergent leadership detection task. Third, human visual behaviour is special because it functions as a social signal and also determines what a person is seeing at a given moment in time. Being able to anticipate human gaze opens up the possibility for machines to more seamlessly share attention with humans, or to intervene in a timely manner if humans are about to overlook important aspects of the environment. We are the first to propose methods for the anticipation of eye contact in dyadic conversations, as well as in the context of mobile device interactions during daily life, thereby paving the way for interfaces that are able to proactively intervene and support interacting humans.Blick, Gesichtsausdrücke, Körpersprache, oder Prosodie spielen als nonverbale Signale eine zentrale Rolle in menschlicher Kommunikation. Sie wurden durch vielzählige Studien mit wichtigen Konzepten wie Emotionen, Sprecherwechsel, Führung, oder der Qualität des Verhältnisses zwischen zwei Personen in Verbindung gebracht. Damit Menschen effektiv während ihres täglichen sozialen Lebens von Maschinen unterstützt werden können, sind automatische Methoden zur Erkennung, Interpretation, und Antizipation von nonverbalem Verhalten notwendig. Obwohl die bisherige Forschung in kontrollierten Studien zu ermutigenden Ergebnissen gekommen ist, bleibt die automatische Analyse nonverbalen Verhaltens in weniger kontrollierten Situationen eine Herausforderung. Darüber hinaus existieren kaum Untersuchungen zur Antizipation von nonverbalem Verhalten in sozialen Situationen. Das Ziel dieser Arbeit ist, die Vision vom automatischen Verstehen sozialer Situationen ein Stück weit mehr Realität werden zu lassen. Diese Arbeit liefert wichtige Beiträge zur autmatischen Erkennung menschlichen Blickverhaltens in alltäglichen Situationen. Obwohl viele soziale Interaktionen in Gruppen stattfinden, existieren unüberwachte Methoden zur Augenkontakterkennung bisher lediglich für dyadische Interaktionen. Wir stellen einen neuen Ansatz zur Augenkontakterkennung in Gruppen vor, welcher ohne manuelle Annotationen auskommt, indem er sich den statistischen Zusammenhang zwischen Blick- und Sprechverhalten zu Nutze macht. Tägliche Aktivitäten sind eine Herausforderung für Geräte zur mobile Augenbewegungsmessung, da Verschiebungen dieser Geräte zur Verschlechterung ihrer Kalibrierung führen können. In dieser Arbeit verwenden wir Nutzerverhalten an mobilen Endgeräten, um den Effekt solcher Verschiebungen zu korrigieren. Neben der Erkennung verbessert diese Arbeit auch die Interpretation sozialer Signale. Wir veröffentlichen den ersten Datensatz sowie die erste Methode zur Emotionserkennung in dyadischen Interaktionen ohne den Einsatz spezialisierter Ausrüstung. Außerdem stellen wir die erste Studie zur automatischen Erkennung mangelnder Verbundenheit in Gruppeninteraktionen vor, und führen die erste datensatzübergreifende Evaluierung zur Detektion von sich entwickelndem Führungsverhalten durch. Zum Abschluss der Arbeit präsentieren wir die ersten Ansätze zur Antizipation von Blickverhalten in sozialen Interaktionen. Blickverhalten hat die besondere Eigenschaft, dass es sowohl als soziales Signal als auch der Ausrichtung der visuellen Wahrnehmung dient. Somit eröffnet die Fähigkeit zur Antizipation von Blickverhalten Maschinen die Möglichkeit, sich sowohl nahtloser in soziale Interaktionen einzufügen, als auch Menschen zu warnen, wenn diese Gefahr laufen wichtige Aspekte der Umgebung zu übersehen. Wir präsentieren Methoden zur Antizipation von Blickverhalten im Kontext der Interaktion mit mobilen Endgeräten während täglicher Aktivitäten, als auch während dyadischer Interaktionen mittels Videotelefonie

Pathology-Related Influences on the VEM: Three Years’ Experience since Implementation of a New Parameter in Phoniatric Voice Diagnostics

The vocal extent measure (VEM) represents a new diagnostic tool to express vocal capacity by quantifying the dynamic performance and frequency range of voice range profiles (VRPs). For VEM calculation, the VRP area is multiplied by the quotient of the theoretical perimeter of a circle with equal VRP area and the actual VRP perimeter. Since different diseases affect voice function to varying degrees, pathology-related influences on the VEM should be investigated more detailed in this retrospective study, three years after VEM implementation. Data was obtained in a standardized voice assessment comprising videolaryngostroboscopy, voice handicap index (VHI-9i), and acoustic-aerodynamic analysis with automatic calculation of VEM and dysphonia severity index (DSI). The complete dataset comprised 1030 subjects, from which 994 adults (376 male, 618 female; 18-86 years) were analyzed more detailed. The VEM differed significantly between pathology subgroups (p<0.001) and correlated with the corresponding DSI values. Regarding VHI-9i, the VEM reflected the subjective impairment better than the DSI. We conclude that the VEM proved to be a comprehensible and easy-to-use interval-scaled parameter for objective VRP evaluation in all pathology subgroups. As expected, exclusive consideration of the measured pathology-related influences on the VEM does not allow conclusions regarding the specific underlying diagnosis

Synchronous visual analysis and editing of RNA sequence and secondary structure alignments using 4SALE

<p>Abstract</p> <p>Background</p> <p>The function of a noncoding RNA sequence is mainly determined by its secondary structure and therefore a family of noncoding RNA sequences is much more conserved on the structural level than on the sequence level. Understanding the function of noncoding RNA sequence families requires two things: a hand-crafted or hand-improved alignment and detailed analyses of the secondary structures. There are several tools available that help performing these tasks, but all of them are specialized and focus on only one aspect, editing the alignment or plotting the secondary structure. The problem is both these tasks need to be performed simultaneously.</p> <p>Findings</p> <p>4SALE is designed to handle sequence and secondary structure information of RNAs synchronously. By including a complete new method of simultaneous visualization and editing RNA sequences and secondary structure information, 4SALE enables to improve and understand RNA sequence and secondary structure evolution much more easily.</p> <p>Conclusion</p> <p>4SALE is a step further for simultaneously handling RNA sequence and secondary structure information. It provides a complete new way of visual monitoring different structural aspects, while editing the alignment. The software is freely available and distributed from its website at <url>http://4sale.bioapps.biozentrum.uni-wuerzburg.de/</url></p

4SALE – A tool for synchronous RNA sequence and secondary structure alignment and editing

BACKGROUND: In sequence analysis the multiple alignment builds the fundament of all proceeding analyses. Errors in an alignment could strongly influence all succeeding analyses and therefore could lead to wrong predictions. Hand-crafted and hand-improved alignments are necessary and meanwhile good common practice. For RNA sequences often the primary sequence as well as a secondary structure consensus is well known, e.g., the cloverleaf structure of the t-RNA. Recently, some alignment editors are proposed that are able to include and model both kinds of information. However, with the advent of a large amount of reliable RNA sequences together with their solved secondary structures (available from e.g. the ITS2 Database), we are faced with the problem to handle sequences and their associated secondary structures synchronously. RESULTS: 4SALE fills this gap. The application allows a fast sequence and synchronous secondary structure alignment for large data sets and for the first time synchronous manual editing of aligned sequences and their secondary structures. This study describes an algorithm for the synchronous alignment of sequences and their associated secondary structures as well as the main features of 4SALE used for further analyses and editing. 4SALE builds an optimal and unique starting point for every RNA sequence and structure analysis. CONCLUSION: 4SALE, which provides an user-friendly and intuitive interface, is a comprehensive toolbox for RNA analysis based on sequence and secondary structure information. The program connects sequence and structure databases like the ITS2 Database to phylogeny programs as for example the CBCAnalyzer. 4SALE is written in JAVA and therefore platform independent. The software is freely available and distributed from the website a

Differential diagnosis of laryngeal spindle cell carcinoma and inflammatory myofibroblastic tumor – report of two cases with similar morphology

BACKGROUND: Spindle cell tumors of the larynx are rare. In some cases, the dignity is difficult to determine. We report two cases of laryngeal spindle cell tumors. CASE PRESENTATION: Case 1 is a spindle cell carcinoma (SPC) in a 55 year-old male patient and case 2 an inflammatory myofibroblastic tumor (IMT) in a 34 year-old female patient. A comprehensive morphological and immunohistochemical analysis was done. Both tumors arose at the vocal folds. Magnified laryngoscopy showed polypoid tumors. After resection, conventional histological investigation revealed spindle cell lesions with similar morphology. We found ulceration, mild atypia, and myxoid stroma. Before immunohistochemistry, the dignity was uncertain. Immunohistochemical investigations led to diagnosis of two distinct tumors with different biological behaviour. Both expressed vimentin. Furthermore, the SPC was positive for pan-cytokeratin AE1/3, CK5/6, and smooth-muscle actin, whereas the IMT reacted with antibodies against ALK-1, and EMA. The proliferation (Ki67) was up to 80% in SPC and 10% in IMT. Other stainings with antibodies against p53, p21, Cyclin D1, or Rb did not result in additional information. After resection, the patient with SPC is free of disease for seven months. The IMT recurred three months after first surgery, but no relapses were found eight months after resurgery. CONCLUSION: Differential diagnosis can be difficult without immunohistochemistry. Therefore, a comprehensive morphological and immunohistochemical analysis is necessary, but markers of cell cycle (apart from the assessment of proliferation) do not help

Investigation of the influence of an oscillation superposition on the wear behaviour in an industrial-like process

In cold forging processes as well as in sheet-bulk metal forming, vast contact stresses result in severe tool wear and thus in tool failures. In order to achieve a sustainable production, a new manufacturing process is developed within the subproject T06 in the transregional collaborative research centre 73 at the Institute of Forming Technology and Machines (IFUM). In this subproject the influence of an oscillation superposition on a forming process is investigated. The new type of sheet-bulk metal forming (SBMF) process manufactures a component with internal and external gearing. Contact normal stresses and thus tool wear could be reduced by applying an oscillation superposition in the main force flow of the machine. To verify the positive results in other processes, the oscillation method is applied to an industrial-like process based on anchor bolt manufacturing of Fischerwerke GmbH & Co. KG. For this purpose, a representative tool system is developed using numerical simulation. The numerical simulation is also used to investigate resulting local contact stresses and relative sliding velocities. Furthermore, cylinder compression tests with and without oscillation superposition are conducted for the workpiece stainless steel 1.4362 (AISI S32304), in order to qualify the reduction of contact stress

Further development of wear calculation and wear reduction in cold forging processes

Tools are of strategic importance for industrial manufacturing processes. Their behaviour has a great influence on the productivity of the process and the quality of the product. A material saving and efficient technique for processing metallic workpieces is cold forging. One major challenge of this production method is the handling of high contact normal stresses in the tool contact, which can lead to severe tool wear. To investigate promising approaches for understanding wear modelling and wear reduction a demonstrator process based on the first stage of a total five-staged cold forging process for the manufacturing of a bolt anchor is considered in the scope of this research. This work aims at the further development of a numerical wear calculation based on an adapted Archard model in order to be able to realistically predict the tool wear in cold forging processes. Therefore, the material characterization of the used workpiece material as well as an investigation of the worn tool dies takes place to validate a numerical wear calculation model. Furthermore, this research addresses a reduction in wear by identifying critical areas and changing the inlet geometry of the investigated demonstrator tool die. This way, conclusions can be drawn about the wear sensitivity during numerical process design, and particularly critical areas can be geometrically modified in terms of the design. © 2021 by the authors. Licensee MDPI, Basel, Switzerland

Accurate experimental determination of gallium K- and L3-shell XRF fundamental parameters

The fluorescence yield of the K- and L3-shell of gallium was determined using the radiometrically calibrated (reference-free) X-ray fluorescence instrumentation at the BESSY II synchrotron radiation facility. Simultaneous transmission and fluorescence signals from GaSe foils were obtained, resulting in K- and L3-shell fluorescence yield values consistent with existing database values(omega_Ga_K=0.515 +- 0.019, omega_Ga_L3=0.013 +- 0.001). For the first time, these standard combined uncertainties are obtained from a properly constructed Uncertainty Budget. These K-shell fluorescence yield values support Bambynek's semi-empirical compilation from 1972: these and other measurements yield a combined recommended value of omega_Ga_K=0.514 +- 0.010. Using the measured fluorescence yields together with production yields from reference Ga-implanted samples where the quantity of implanted Ga was determined at 1.3% traceable accuracy by Rutherford backscattering spectrometry, the K-shell and L3-subshell photoionization cross sections at selected incident photon energies were also determined and compared critically with the standard databases.Comment: 17 pages, 6 figure